AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.250002	2.73
	2_DecisionTree	Decision Tree	logloss	0.222161	133.45
	3_Default_Xgboost	Xgboost	logloss	0.201696	129.52
	4_Default_NeuralNetwork	Neural Network	logloss	0.235937	144.3
	5_Default_RandomForest	Random Forest	logloss	0.221147	603.16
	3_Default_Xgboost_GoldenFeatures	Xgboost	logloss	0.201963	129.47
	2_DecisionTree_GoldenFeatures	Decision Tree	logloss	0.219555	129.2
	4_Default_NeuralNetwork_GoldenFeatures	Neural Network	logloss	0.236634	149.24
the best	Ensemble	Ensemble	logloss	0.201486	8.39

AutoML Performance

AutoML Performance Boxplot

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.8 seconds

Metric details

	score	threshold
logloss	0.250002	nan
auc	0.5	nan
f1	0.128392	0.06173
accuracy	0.0686	0.06173
precision	0.0686	0.06173
recall	1	0.06173
mcc	0	0.06173

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.250002	nan
auc	0.5	nan
f1	0.128392	0.06173
accuracy	0.0686	0.06173
precision	0.0686	0.06173
recall	1	0.06173
mcc	0	0.06173

Confusion matrix (at threshold=0.06173)

	Predicted as 0	Predicted as 1
Labeled as 0	0	55884
Labeled as 1	0	4116

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: entropy
• max_depth: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

132.4 seconds

Metric details

	score	threshold
logloss	0.222161	nan
auc	0.749963	nan
f1	0.309402	0.126857
accuracy	0.922717	0.32701
precision	0.363111	0.32701
recall	1	0.00126478
mcc	0.254071	0.126857

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.222161	nan
auc	0.749963	nan
f1	0.229606	0.32701
accuracy	0.922717	0.32701
precision	0.363111	0.32701
recall	0.167881	0.32701
mcc	0.210869	0.32701

Confusion matrix (at threshold=0.32701)

	Predicted as 0	Predicted as 1
Labeled as 0	54672	1212
Labeled as 1	3425	691

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree_GoldenFeatures

<< Go back

Decision Tree

• n_jobs: -1
• criterion: entropy
• max_depth: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

128.2 seconds

Metric details

	score	threshold
logloss	0.219555	nan
auc	0.751138	nan
f1	0.328546	0.126878
accuracy	0.927117	0.346464
precision	0.326457	0.126878
recall	1	0.00198572
mcc	0.278758	0.126878

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.219555	nan
auc	0.751138	nan
f1	0.0581521	0.346464
accuracy	0.927117	0.346464
precision	0.256167	0.346464
recall	0.0327988	0.346464
mcc	0.0698508	0.346464

Confusion matrix (at threshold=0.346464)

	Predicted as 0	Predicted as 1
Labeled as 0	55492	392
Labeled as 1	3981	135

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

128.3 seconds

Metric details

	score	threshold
logloss	0.201696	nan
auc	0.822041	nan
f1	0.347872	0.175639
accuracy	0.93075	0.44077
precision	0.467446	0.44077
recall	1	0.000553567
mcc	0.295999	0.175639

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.201696	nan
auc	0.822041	nan
f1	0.11877	0.44077
accuracy	0.93075	0.44077
precision	0.467446	0.44077
recall	0.0680272	0.44077
mcc	0.15845	0.44077

Confusion matrix (at threshold=0.44077)

	Predicted as 0	Predicted as 1
Labeled as 0	55565	319
Labeled as 1	3836	280

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost_GoldenFeatures

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

128.4 seconds

Metric details

	score	threshold
logloss	0.201963	nan
auc	0.82174	nan
f1	0.34556	0.174912
accuracy	0.930417	0.438242
precision	0.450751	0.438242
recall	1	0.000616663
mcc	0.293573	0.145551

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.201963	nan
auc	0.82174	nan
f1	0.114528	0.438242
accuracy	0.930417	0.438242
precision	0.450751	0.438242
recall	0.0655977	0.438242
mcc	0.151817	0.438242

Confusion matrix (at threshold=0.438242)

	Predicted as 0	Predicted as 1
Labeled as 0	55555	329
Labeled as 1	3846	270

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

143.1 seconds

Metric details

	score	threshold
logloss	0.235937	nan
auc	0.694712	nan
f1	0.211239	0.116338
accuracy	0.66195	0.116338
precision	0.125747	0.116338
recall	1	7.56648e-05
mcc	0.170984	0.108263

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.235937	nan
auc	0.694712	nan
f1	0.211239	0.116338
accuracy	0.66195	0.116338
precision	0.125747	0.116338
recall	0.659864	0.116338
mcc	0.169553	0.116338

Confusion matrix (at threshold=0.116338)

	Predicted as 0	Predicted as 1
Labeled as 0	37001	18883
Labeled as 1	1400	2716

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 4_Default_NeuralNetwork_GoldenFeatures

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

148.0 seconds

Metric details

	score	threshold
logloss	0.236634	nan
auc	0.680723	nan
f1	0.197464	0.0975385
accuracy	0.551717	0.0975385
precision	0.112555	0.0975385
recall	1	4.59745e-06
mcc	0.170441	0.0975385

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.236634	nan
auc	0.680723	nan
f1	0.197464	0.0975385
accuracy	0.551717	0.0975385
precision	0.112555	0.0975385
recall	0.803936	0.0975385
mcc	0.170441	0.0975385

Confusion matrix (at threshold=0.097538)

	Predicted as 0	Predicted as 1
Labeled as 0	29794	26090
Labeled as 1	807	3309

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

602.1 seconds

Metric details

	score	threshold
logloss	0.221147	nan
auc	0.761092	nan
f1	0.308757	0.205884
accuracy	0.9292	0.333517
precision	0.424138	0.333517
recall	1	0.00588364
mcc	0.253586	0.205884

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.221147	nan
auc	0.761092	nan
f1	0.148014	0.333517
accuracy	0.9292	0.333517
precision	0.424138	0.333517
recall	0.0896501	0.333517
mcc	0.170613	0.333517

Confusion matrix (at threshold=0.333517)

	Predicted as 0	Predicted as 1
Labeled as 0	55383	501
Labeled as 1	3747	369

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	3
3_Default_Xgboost_GoldenFeatures	2

Metric details

	score	threshold
logloss	0.201486	nan
auc	0.822858	nan
f1	0.347314	0.175358
accuracy	0.930417	0.438295
precision	0.450751	0.438295
recall	1	0.000620673
mcc	0.297133	0.145456

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.201486	nan
auc	0.822858	nan
f1	0.114528	0.438295
accuracy	0.930417	0.438295
precision	0.450751	0.438295
recall	0.0655977	0.438295
mcc	0.151817	0.438295

Confusion matrix (at threshold=0.438295)

	Predicted as 0	Predicted as 1
Labeled as 0	55555	329
Labeled as 1	3846	270

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back